OPEN 3 ACCESS Freely available online 



•0-PLOS I o-^E 



Problem-Based Training Improves Recognition of Patient 
Hazards by Advanced Medical Students during Chart 
Review: A Randomized Controlled Crossover Study 

Friederike Holderried\ Daniel Heine^, Robert Wagner^, Moritz Mahling^ Yelena Fenik^ Anne Herrmann- 
Werner^*, Reimer Riessen^, Peter Weyrich^, Stephan Zipfel'*, Nora Celebi^ 

1 University of Tuebingen, Medical School, Office of the Dean of Student Affairs, Tuebingen, Germany, 2 University Hospital of Tuebingen, Department for 
Anaesthesiology, Tuebingen, Germany, 3 University of Tuebingen, Medical School, Tuebingen, Germany, 4 University Hospital of Tuebingen, Department for Internal 
Medicine VI, Psychosomatic Medicine, Tuebingen, Germany, 5 University Hospital of Tuebingen, Medical Intensive Care Unit, Tuebingen, Germany, 6 University Hospital of 
Tuebingen, Department for Internal Medicine IV, Angiology, Endocrinology, Nephrology and Clinical Chemistry, Tuebingen, Germany, 7 Nephrology Center Marienpark, 
Stuttgart, Germany 



Abstract 

Background: Patient chart review is the gold standard for detection of potential patient hazards (i.e. medication errors or 
failure to follow up actionable results) in both routine clinical care and patient safety research. However, advanced medical 
students' ability to read patient charts and to identify patient hazards is rather poor. We therefore investigated whether it is 
possible to teach advanced medical students how to identify patient hazards independent of context (i.e. cancer versus 
cardiac failure) in patient charts. 

Methods: AW fifth-year medical students in one semester (n = 123) were randomized into two groups. One group (IC) 
received a patient chart review-training first and then a control-intervention and the other group (CI) received the control- 
intervention first and then the patient chart review-training. Before and after the teaching sessions, students reviewed 
different scenarios with standardized fictional patient charts containing 12 common patient hazards. Two blinded raters 
rated the students' notes for any patient hazard addressed in the notes using a checklist. The students were blinded to the 
study question and design. There was no external funding and no harm for the participating students. 

Results: k total of 35 data sets had to be excluded because of missing data. Overall, the students identified 17% (IQR 8- 
29%) of the patient hazards before the training and 56% (IQR 41-66%) of the patient hazards after the training. At the 
second assessment students identified more patient hazards than at the first. They identified even more in the third. The 
effect was most pronounced after the patient chart review training (all p<.01). 

Conclusion: Patient chart review exercises and problem-based patient chart review training improve students' abilities to 
recognize patient hazards independent of context during patient chart review. 
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Introduction 

Some common patterns constitute a large fraction of patient 
hazards. According to the data of the National Patient Safety 
Benchmarking Center, five most frequent medical errors and 
adverse events (surgical errors, medication errors, medical errors, 
patient falls, nosocomial infections) account for 67% of medical 
errors and for 81% of the costs attributable to medical errors [1]. 
In internal medicine, medication problems and diagnostic errors 
are frequent patient hazards [2]. 

Patient chart review is considered to be the gold standard for the 
identification of many common patient hazards, especially 
diagnostic problems (delayed, missed, wrong diagnosis, failure to 
follow up on actionable test results), medication problems 



(overmedication, undermedication, wrong dosage, medication 
contraindications, unauthorized medication, drug-drug interac- 
tions), nosocomial infections and problematic fluid/diet-manage- 
ment [3,4,5,6,7,8,9]. In patient safety research, patient charts are 
retrospectively reviewed by a nurse or physician specifically 
trained for this task [10]. 

Since patient chart review should be routinely performed during 
every ward round, it has the potential to be a major tool for the 
promotion of patient safety. Previous investigations have shown 
that advanced medical students' ability to read and interpret 
patient charts is rather poor: Nikendei et al. performed ward 
round exercises with final year medical students and found that the 
students performed only 49% of the expected tasks during patient 
chart review, prescribing and documentation [11]. In a previous 
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study by our group, 5 year medical students identified only 20% 
of the patient hazards in fictional patient charts, irrespective of the 
number of weeks they spent in clerkships in internal medicine [12]. 
In a study by Heaton, which included 2413 medical students and 
recently graduated physicians, many participants stated that they 
learned to apply their pharmacology knowledge to real clinical 
situations only opportunistically during clinical rotations and the 
vast majority felt inadequately prepared for safe and effective 
usage of drugs [13]. Thus there might be a gap in transferring the 
knowledge acquired during medical school classes to real patient 
care. 

Complex medical tasks like patient chart review and ward 
rounds are only seldom taught in medical school; students report 
performing them only sporadically during clerkships [14,15]. In a 
study by de Feijter et al., medical students reported being torn 
between the acquisition of skills through practicing on real patients 
and delivering safe patient care, which led to an avoidance of 
performing tasks such as prescribing [16]. Medical students and 
novice graduates are only rarely supervised by experienced staff 
[17,18]. 

Since young doctors have to perform ward rounds and patient 
chart review from day one of their graduation, medical education 
should address patient chart review and common patient hazards. 
In this study, we simulated the medical routine through complex 
patient management problems, thus we refer to the lessons as 
"problem-based training". We did not use the checklist used in 
patient safety research but concentrated on common and easy to 
detect patient hazards instead. 

Using a randomized, controlled cross-over-study, we investigat- 
ed whether a structured problem-based patient chart review 
training improves the identification of common patient hazards, 
that is, diagnostic problems, medication problems, inadequate 
monitoring, and nosocomial infections, independent of context 
during patient chart reviews by advanced medical students. 

Methods 

Study Design and Participants 

We conducted a prospective randomized trial with a cross-over 
design in order to identify potential superiority of the problem- 
based patient chart review training over the control intervention 
(see Figure 1). All 5* year medical students participating in the 
mandator)'- int(;rnal m('dicine class (n = 123) were randomized into 
two equally large groups using a list of random numbers according 
to the following protocol: a list of five digit-random numbers was 
written down in three columns. Every student was assigned to one 
line, so that every student was pseudonymized with three random 
numbers. ()n(; for the first, one for the second and one for the third 
assessment. If the second digit of the first random number was 
odd, the student was assigned to the intervention - control group, 
if the second digit was even, he or she was assigned to the control - 
intervention group. For the rating, all assessment-patient charts 

^ Oi-X-02-C-03 

R,n = 123<: 

O1-C-O2-X-O3 

Figure 1. Study design. R = Randomization of 123 5* year medical 
students, 0 = Observation (review of a fictional patient chart with 
twelve common patient hazards), X = Intervention (training on patient 
chart review and patient safety), C = Control-Intervention (ultrasound 
and Skills lab-training). 
dol:1 0.1 371/journal.pone.00891 98.g001 



were ordered according to the random numbers. The first rater 
rated them from top down, the second from bottom-up. 

One group (intervention - control group, IC) received 1 5 hours 
of training in patient chart review and patient safety followed by a 
15-hour control- intervention on abdominal ultrasound and skills- 
lab training. The other group (control - intervention, CI) received 
the control intervention first and the problem-based patient chart 
review training afterwards. While participation in the training was 
mandator)', the assessment was not. The students were free to 
decide, whether they submitted their assessment forms for the 
study or not. The students could withdraw from the study anytime 
without giving any reasons and without effect on their grading or 
other disadvantages. The mecUcEil students were blinded to the 
study question and the study design. 

The study design is shown in figure 1. Before and after the 
interventions (problem-based patient chart review training, X, 
skUls-lab training, C) the students were asked to review three 
fictional patient charts (observations, Oi_3) in 30 minutes. The 
patient charts were standardized for twelve common patient 
hazards and permutated, so that every scenario served as first, 
second and third observation with the same frequency. The 
students were asked to address any problem they found in the 
patient chart but were not told that the focus of the assessment was 
on detection of patient hazards. 

The students were asked to prepare for the main diagnoses 
(further details in the Assessment Section) three days in advance of 
each observation period, since we did not want to test their general 
knowledge about diseases. There were no changes in the study 
protocol after the trial commenced. 

Setting 

The study was conducted at the University of Ttlbingen, 
Germany, a medical s[:hool with a traditional 6-year curriculum. 
The first two years are dedicated to basic sciences (e.g. anatomy, 
physiology, biochemistry). The core clinical medical training takes 
place during years 3-5, the 6* (final) year comprises three four- 
month internships in internal medicine, surgery and an elective. 
Our students were all in the fifth year, so they completed most of 
their clinical training (pharmacology classes, lessons on internal 
medicine, etc.). 

Assessment 

Patient management problems have a good face validity and 
have been used successfully for teaching and assessment [19]. The 
actual patient hazard patterns used in this study were based on 
recent publications on patient safety [2,6,20,21,22]. 

We thus prepared fictional patient charts with three different 
scenarios (cellulitis in an immunosuppressed patient, atrial 
fibrillation in hyperthyroidism, patient with advanced cancer in 
pain) and standardized the patient charts with twelve common 
patient hazards: 

- One indicated medication missing 

- One medication not indicated 

- One medication with the wrong dosage 

- One risk situation for a non-authorized medication 

- One side effect of the medication 

- One contraindication against a medication 

- One incidental diagnostic finding 

- A missing diagnostic tests for the main problem 

- Missing monitoring 

- One infectious complication 
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Table 1. Example of a case vignette with twelve common 


patient hazards. 




Scenario 


Cellulitis in patient with a renal graft, penicillin allergy 


Drug missing 


Thromboembolic prophylaxis 


Not-indicated medication 


Penicillin 


Medication with wrong dosage 


Immunosuppression 10-fold overdosed 


contraindication against a drug 


Penicillin allergy 


Side effect of the medication 


Steroid diabetes 


Risk situation for non-authorized medication 


Pain killer 


Incidental finding 


Atypical naevus 


Missing diagnostic test for the main problem 


Blood culture 


Missing monitoring 


Tacrolimus-level 


Infectious complication 


Not used central line catheter 


Wrong Diet/fluid management 


Exsiccosis 


Incomplete Documentation 


Anemia 


doi:1 0.1 371/journal.pone.00891 98.t001 



— Wrong diet/fluid management 

— Incomplete documentation 

Table 1 shows an exemplary case vignette as used in the study. 

The actual patient hazards were different in the three scenarios. 
The scenarios were developed by two physicians and approved for 
face validity by another four physicians. We constructed the 
fictional patient charts so that they would resemble the real patient 
charts used in our tertiary care hospital. They comprised: a cover 
sheet with patient data, the history, main diagnosis and 
comorbidities and results of the main examinations, the actual 
patient chart recording the patient's vital signs, the medication, 
nursing notes and the laboratory results. 

The students were asked to review the patient chart and amend 
documentation, prescribe medication, and order diagnostic tests as 
they saw fit for any problem they identified. We only assessed, 
whether the problem was identified, not whether subsequent 
measures were appropriate. The prescription and documentation 
forms were subsequently rated by two blinded raters using a 
checklist for every problem addressed. The scores were calculated 
as number of problems found divided through total number of 
problems in the patient chart. 

Intervention 

According to Kolb and Fry, an effective training comprises four 
steps: concrete experience, observation and reflection, the 
formation of abstract concepts, and testing in new situations 
[23]. This was the conceptual framework for the patient chart 
review-training. The first assessment on patient chart review, 
which was done before any intervention, served as the concrete 
experience. Reflection and formation of abstract concepts was 
supposed to be facilitated by the theoretical input and discussion of 
the cases during the patient chart review training. The assessment 
in a different context after the problem-based patient chart review 
training (Oj and O3 for the IC, O3 for the CI) served as the testing 
in new situations. 

The patient chart review-training lasted one week and 
comprised: 

— A patient chart reading exercise in small groups (90 minutes) 

— A lecture on common patient hazards (45 minutes) 



- Preparation of seminar papers on various topics concerning 
patient management (e.g. indications for thromboembolic 
prophylaxis, optional medication) 

- Three group exercises to prepare standardized fictional patient 
charts containing common patient hazards in small groups (90 
minutes each) 

- Three lectures discussing the prepared standardized fictional 
patient charts with a lecturer (90 minutes each) 

The control-intervention also lasted one week and comprised 
training in the skiUs-lab on abdominal ultrasound (8 h) and 
puncture-techniques (central line catheter, pleura, arterial punc- 
ture, bone marrow) on a manikin (7 h). 

Statistics 

We used JMP 9.0 (SAS Institute Inc, Gary, NC, USA) to 
compute the results. Calculations were based on mean rater 
evaluations. Since the values of the identified patient hazards were 
skewed, they were compared using the Wilcoxon rank-sum test 
and displayed as mean and interquartile ratio (IQR). We 
performed a power analysis (G*Power software, Erdfelder, Faul, 
& Buchner, 1996, Dtlsseldorf, Germany) based on a previous 
investigation on prescription errors [24]. 

Our null hypothesis was defined as no difference between the 
percentage of recognized patient hazards after the problem-based 
intervention compared to the control-intervention. 

Ethics 

The study was conducted in accordance with the Declaration of 
Helsinki (revised form, Seoul 2008). The study protocol was 
approved by the local Ethics Committee, decision number 488/ 
20 IDA. The medical students gave written consent. Study 
participation was voluntary; the students could withdraw without 
giving any reason. No harms or adverse effects were encountered. 

Results 

According to the power analysis we could detect a difference of 
10% with n = 74, a = .05 and SD= .15. 
The overall interrater-reliability was .96. 
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Table 2. Characteristics of the study participants. 







IC 


CI 


Age 


26 ±3 years 


26 ±3 years 


Gender 


13 male, 30 female 


23 male, 22 female 


Previous education 


2 nurse/paramedic, 5 aide on the ward, 36 none 


4 nurse/paramedic, 5 aide on the ward, 36 none 


Previously performed patient chart reviews 


40 never, 3 1-5 times 


40 never, 3 1-5 times, 2>5 times 


N 


43 


45 
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We had to exclude 35 students due to missing data since they 
did not complete all three assessments. The characteristics of the 
remaining 88 students are shown in Table 2. 

Overall, the students identified 17% (IQR 8-29%) of the patient 
hazards at O] 56% (IQR 41-66%) of the patient hazard.s at O3, 
p<.0001. 

At Oi, when no group had patient chart review training, the 
scores of the groups did not differ (IC mean 2 1 % (IQR 8-33%) vs. 
CI mean 17% (IQR 8-2 1 %), p = .95). At O2, when the IC already 
had patient chart review-training but the CI not, the scores 
differed significantly (IC mean 42% (IQR 33-58%) vs. CI mean 
29% (IQR 21-42%), p<.0001). At O3, when both groups had the 
problem-based patient chart review training, there was again no 
significant difference (IC mean 54% (IQR 43-66%) vs. CI mean 
63% (IQR 42-72%), p = .26). At O, the .students identified 
significantly more patient hazard than during Oi and even more 
during O3. The effect was more pronounced after problem-based 
patient chart review training but also significant after the control 
intervention, although the assessments comprised different sce- 
narios (all p<.01) (see Figure 2). 

Discussion 

In our study, the skill of advanced medical students to identify 
patient hazards during patient chart review improved after 
problem-based patient chart review training. The effect was partly 
context-independent, so it was possible to train the students on a 



* * 




□ IC 

□ 01 



Figure 2. Patient hazards Identified during patient chart 
review. IC: intervention - control-group (first patient chart review- 
training, then control-Intervention), CI: control - Intervention-group 
(first control-intervention, then patient chart review-training). Oi_3: 
observations, review of a standardized fictional patient chart with 
twelve common patient hazards. A star indicates a significant difference 
(all p<.01). 
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case with advanced cancer and the students were able to apply the 
skill to a case with atrial fibrillation and hyperthyroidism. The 
main reason for this phenomenon might be, that we taught the 
students to look for patterns that may impose patient hazards, and 
most of these patters do not necessarily require context-specific 
knowledge (i.e. check every drug for adequate dosing) or refer to 
knowledge already covered during previous classes (e.g. drug-drug- 
interactions taught in their pharmacology classes). In addition, the 
raised awareness for patient safety may have had an influence on 
the way the students review the patient charts. 

Moreover, the students improved most when they had the 
problem-based patient chart review training, but also identified 
more patient hazards after the control intervention only. 

This corresponds with a previous study on prescription errors, 
where the ability to critically appraise medication and to modify it 
to a specific clinical context was improved independent of context 
with a problem-based training [25]. In a previous study we 
investigated whether clerkships or previous training improve the 
identification of patient hazards during patient chart review and 
found that clerkships alone did not have an effect, while students, 
who reported to have performed at least one patient chart review 
themselves, identified significantly more patient hazards than 
students who had never performed a patient chart review [24] . So 
the improvement after the control-intervention might actually be 
an improvement due to the practice in patient chart review during 
the observations. While the students might be able to solve the 
tasks when presented with bare facts, finding the patterns in a 
patient chart bursting with information constitutes a much bigger 
chaUenge, which reflects the daily work of physicians in a more 
realistic manner. 

Our study has several limitations. Since study participation was 
voluntary and the interventions and assessments lasted two weeks, 
we had to exclude 28% of the students due to missing data, so 
there might be a distortion in the study sample. The students only 
reviewed fictional standardized patient charts, not real patient 
charts. While we tried to balance the patient hazards carefully, the 
students might have stopped looking for problems after identifying 
several patient hazards, since they did not expect so many patient 
hazards in one patient chart. In addition, we cannot exclude that 
the students IC group discussed the problem-based patient chart 
review training with the Cl-group at any point, although we asked 
them not to. 

While the medical students were blinded to the study question 
and the study design, the instructor was not. The identification is 
only a procedural outcome and it has yet to be proven that the 
identification of patient hazards can actually avert patient harm. 
We cannot exclude that a simple lesson on patient safety alone 
might also have improved the ability to identify patient hazards, 
since the students received this lesson during the patient chart 
review training. 
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Further research is warranted on the ability of medical students 
and novice graduates to read patient charts and to identify patient 
hazards and whether the training has an effect on patient safety. 

In conclusion, a problem-based patient chart review training 
and patient chart review practice together improve the skills of 
advanced medical students in identifying patient hazards during 
patient chart review and thus might probably contribute to patient 
safety. 
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